RESEARCH

The Role of Natural Image Statistics in Modulating Early VEP Components

The contrast response function of early visual evoked potentials (VEPs) elicited by sinusoidal gratings is known to exhibit characteristic potentials closely associated with the processes of parvocellular and magnocellular pathways.  Specifically, the N1 component has been linked with parvocellular processes, while the P1 component has been linked with magnocellular processes.  However, little is known regarding the response properties of the N1 and P1 components during the processing and encoding of complex (i.e., broadband) stimuli such as natural scenes.  Here, we examine how established physical characteristics of natural scene imagery modulate the N1 and P1 components in humans by providing a systematic investigation of component modulation as visual stimuli are gradually built up from simple sinusoidal gratings to highly complex natural scene imagery.  The results suggest that the relative dominance in signal output of the N1 and P1 components is dependent on spatial frequency luminance contrast for simple stimuli up to natural scene imagery possessing few edges.  However, such a dependency shifts to a dominant N1 signal for natural scenes possessing abundant edge content, and operates independently of spatial frequency luminance contrast.

A Cortical Locus for Overlay Suppression Revealed with tDCS

Human contrast sensitivity for Gabor targets is largely suppressed when superimposed on a Gabor mask of the same spatial frequency (SF) and orientation (referred to as overlay suppression, OS), with suppression being broadly tuned to orientation (~130° bandwidth) and spatial frequency (~3 octave bandwidth) (Petrov, Carandini, & McKee, 2005).  Further, numerous behavioral and neurophysiological experiments have suggested that the suppressive signals leading to OS originate from the initial LGN inputs to V1.  However, Kim, Haun, and Essock (2010) have reported anisotropic contrast sensitivity thresholds for differently oriented Gabor targets when masked by broadband noise.  Their results suggest a much narrower orientation tuning for OS than would be expected if it originates from initial LGN inputs to V1, and imply that another cortical operation is active when overlay masks are broadband. This project sought to explore the above notion using transcranial direct current stimulation (tDCS).  Since tDCS is only effective at modulating cortical function, we hypothesized that tDCS would influence suppression for OS with broadband stimuli but not for OS with narrowband stimuli which may result from sub-cortical signals. Human participants performed the task foveally with tDCS (anodal and cathodal) and without (i.e., baseline).  The results showed that tDCS resulted in a large anisotropic reduction in OS when the masks were broadband, but not when they were narrowband.  This supports the hypothesis that OS has a cortical locus when the overlay masks are broadband in SF and orientation.

The Influence of Implicit and Explicit Racial Prejudice on Fixation Behavior

It is well known that humans make use of internalized schemas to guide their looking behavior.  Specifically, the majority of what we look at (i.e., fixate) can be directed by our expectations of where to find useful information.  Interestingly, our internalized schemas for different human faces are known to reflect systematic biases that are consistent with one’s degree of racial prejudice. However, it has yet to be objectively demonstrated whether humans possessing varying amounts of racial prejudice (thus operating on differently biased face-schemas) will look at faces differently.  Further, since humans are known to also vary in prejudice type (i.e., prejudice carried implicitly or explicitly), we don’t know whether individuals high in implicit or explicit prejudice examine faces differentially.  This project sought to provide an answer to the above questions.  Participants varying in their degree of implicit or explicit prejudice viewed Black or White faces while having their gaze behavior recorded with an eye-tracker.  Data were analyzed for differing levels and type of racial prejudice in terms of fixation pattern biases (i.e., whether certain facial regions were fixated longer and more frequently than others), as well as global scan strategy (i.e., the sequence of fixations from one facial region to the next).  The results show that, regardless of prejudice type (i.e., implicit or explicit), participants high in racial prejudice examine faces differently than those low in racial prejudice by showing a stronger tendency to ‘avoid’ fixating critical face features, such as the eyes.  Moreover, participants classified as high in implicit racial prejudice examined faces differently than those high in explicit prejudice.  It therefore seems that differently prejudiced individuals gather face information (via fixation patterns and global scan strategies) consistent with their specific internal biases regarding racial prejudice.

Extracting the Internal Representation of Human Faces

Much of the debate surrounding the precise functional role of brain mechanisms implicated in the processing of human faces can be explained when considering that studies into early-stage neural representations of the spatial arrangement of facial features are potentially contaminated by “higher level” cognitive attributes associated with human faces.  One way to bypass such attributes would be to employ ambiguous stimuli that are not biased toward any particular object class and analyze neural activity in response to those stimuli in a manner similar to traditional reverse correlation for mapping visual receptive fields.  Accordingly, we sought to derive whole face representations directly from neural activity in the human brain using electroencephalography (EEG).  We presented ambiguous fractal noise stimuli to human participants and asked them to rate each stimulus along a “face not present” to “face present” continuum while simultaneously recording EEG’s.  All EEG’s were subjected to a time-frequency analysis near 170ms (negative amplitudes near 170ms post-stimulus onset have been linked to early face processing) for five different frequency bands (delta, theta, alpha, beta, and gamma) on a trial-by-trial basis, independent of the behavioral responses.  Images containing apparent face-like structure were obtained for theta through gamma frequency bands for strong negative amplitudes near 170 ms post-stimulus onset.  The presence of the face-like structure in the spatial images derived from brain signals was objectively verified using both Fourier methods and trained neural networks.  The results support the use of a modified reverse correlation technique with EEG as a non-biased assessment of brain processes involved in the complex integration of spatial information into objects such as human faces.

Rapid Scene Categorization

We investigated whether higher-order image statistics, known to be carried by the Fourier phase spectrum, are sufficient to affect scene gist recognition.  Experiment 1 compared the scene gist masking strength of four masking image types that varied in their degree of second and higher-order relationships: normal scene images, scene textures, phase-randomized scene images, and white noise.  Masking effects were largest for masking images that possessed significant higher-order image statistics (scene images and scene textures) when compared to masking images that did not (phase-randomized scenes and white noise), with scene image masks yielding the largest masking effects.  A control study eliminated all differences in the second order statistics of the masks, while maintaining differences in their higher-order statistics, by comparing masking by scene textures versus their phase-randomized versions, and showed that the former produced significantly stronger gist masking.  Experiments 2 and 3 were designed to test whether conceptual masking could account for the differences in the strength of the scene texture and phase-randomized masks used in Experiment 1, and revealed that the recognizability of scene texture masks explained just 1% of their masking variance.  Together, the results suggest that 1) masks containing the higher-order statistical structure of scenes are more effective at masking scene gist processing than masks lacking such structure, and 2) much of the disruption of scene gist recognition that one might be tempted to attribute to conceptual masking is due to spatial masking.

Retinotopic Organization of Spatial Frequency in Amblyopia

The processing deficit in amblyopia is not restricted to just high spatial frequencies but also involves low-medium spatial frequency processing, for suprathreshold stimuli with a broad orientational bandwidth. This is the case in all three forms of amblyopia; strabismic, anisometropic and deprivation. Here we use both a random block design and a phase-encoded design to ascertain 1. The extent to which fMRI activation is reduced at low-mid spatial frequencies in different visual areas, 2. how accurately spatial frequency is mapped across the amblyopic cortex. We report a loss of function to suprathreshold low-medium spatial frequency stimuli that involves more than just area V1, suggesting a diffuse loss in spatial frequency processing in a number of different cortical areas. An analysis of the fidelity of the spatial frequency cortical map reveals that many voxels lose their spatial frequency preference when driven by the amblyopic eye, suggesting a broader tuning for spatial frequency for neurons driven by the amblyopic eye within this low-mid spatial frequency range.

Structural Alignment in Human Face Recognition and Discrimination

We investigated the processes underlying the discrimination and recognition of human faces as a function of spatial phase alignment to assess whether face processing can be understood in terms of the amplitude spectrum alone.  Specifically, we varied the amount of aligned Fourier phase in different regions of the face frequency spectrum and argue that the properties of the underlying neural processes are best understood in terms of the number of phase-alignments as opposed to octave bandwidths.  Additionally, we observed performance differences for face discrimination tasks compared to face recognition tasks.  For face recognition, our results show that a narrower range of phase alignment is needed for face frequencies near 9cpf when compared to 3cpf and 27cpf, thereby supporting the notion of a critical frequency for face recognition.  However, for face discrimination where participants were required to discriminate between an average face and different unique faces along a face morph continuum, performance depended on a fixed signal-to-noise ratio of phase-alignment within a contiguous range of face frequencies (termed critical band of phase alignments), regardless of the central face frequency of that range within the face frequency spectrum when compared to non phase randomized control thresholds. 

The Development of the Visual System is Gradual for Natural Scenes

Accumulating evidence suggests that the adult visual system is optimally tuned for processing the spatial properties of natural scenes (1∕ f^α amplitude spectra). It is also documented that different aspects of spatial vision (e.g., acuity and spatial contrast sensitivity) develop at different rates and some do not become mature until late childhood. Here, we compared natural scene perception in children aged 6, 8, and 10 years (n = 16 per age) and in adults (mean age = 23). A same-different task combined with a staircase procedure measured thresholds for discriminating changes in the slope of the amplitude spectra of natural scene stimuli with reference α’s of 0.7, 1.0, or 1.3. First, consistent with previous studies, adults were least sensitive for the shallowest  α (i.e., 0.7) and most sensitive for the steepest α (i.e., 1.3). Second, a 4 (age group) X 3 (reference α’s) repeated measures ANOVA revealed a significant interaction (p < 0.01). Post-hoc analyses of the interaction indicated no difference in threshold among any of the age groups for the α of 0.7. However, the 6- and 8-year-olds had significantly higher discrimination thresholds compared to the 10-year-olds and adults for α’s of 1.0 and 1.3. Finally, the 10-year-olds’ thresholds did not differ significantly from those of adults for any of the α’s tested. These data suggest that sensitivity to detecting changes in the spatial characteristics of natural scenes during childhood may not be optimally tuned to the statistics of natural images until about 10 years of age.

Free-viewing of Natural Scenes Following Brief Adaptation

While the global structure of natural scene images is known to exhibit relatively stable biases across a large number of images with respect to the 1/ƒα amplitude relationship and horizontal-vertical amplitude biases, the local structure within given images varies considerably as a function of distance. It has recently been shown that when macaque or rhesus monkeys free-view natural scene images, a largely orthogonal difference between the orientation statistics of sequential fixations was observed for long saccades. Here, we sought to extend this paradigm to human observers by examining the local orientation statistics or local amplitude spectrum slope of fixated natural or synthetic image regions during free-viewing following brief adaptation to narrow or broadband orientation patterns or isotropic visual noise patterns possessing different amplitude spectrum slopes. The data showed that while human observers tended to fixate image regions containing orientation statistics approximately orthogonal to the adapting patterns, this effect was dependent on global structural sparseness of the free-viewed images, with the strongest effects occurring for scenes possessing sparse structure. A similar effect was observed when the adapting pattern possessed a steep or shallow amplitude spectrum slope, only this effect was less dependent on the global structural sparseness of the free-viewed images. Taken together, these findings argue that when specific processing units of V1 are briefly adapted, the selection of content at subsequent fixation regions in natural images possessing sparse structure is influenced by adaptation.

Structural Sparseness and Spatial Phase Alignment in Natural Scenes

The Fourier phase spectrum plays a central role regarding where in an image contours occur, thereby defining the spatial relationship between those structures in the overall scene.  Only a handful of studies have demonstrated psychophysically the relevance of the Fourier phase spectrum with respect to human visual processing, and none have demonstrated the relative amount of local cross-scale spatial phase alignment needed to perceptually extract meaningful structure from an image.  Here, we investigated the relative amount of spatial phase alignment needed for humans to perceptually match natural scene image structures at three different spatial frequencies (3cpd, 6cpd, and 12cpd) as a function of the amount of structures within the image (i.e., “structural sparseness”).  The results showed that 1) the amount of spatial phase alignment needed to match structures depends on structural sparseness, with a bias for matching structures at 6cpd, and 2) the ability to match partially phase randomized images at a given spatial frequency is independent of structural sparseness at other spatial frequencies.  The findings of the current study are discussed in terms of a network of feature integrators in the human visual system.

Foveal and Parafoveal Processing of Natural Scenes

A number of studies have investigated whether human visual performance can be related to the general form of the amplitude spectra (i.e., 1/ƒ^α) of natural scenes.  Here, it is argued that there are some discrepancies in the data between some of those studies, and that one possible explanation for the discrepancies may be related to differences in methodology (e.g., stimuli presented to the fovea opposed to the parafovea).  We sought to resolve some of the discrepancies with two psychophysical paradigms involving a discrimination with visual noise and natural scene image patches presented to the fovea or parafovea.  Fovea—parafovea threshold differences were apparent for stimuli possessing a values less than 1.0, with the parafovea typically showing highest thresholds for reference as in the 0.74 – 0.85 range.  Both fovea and parafovea thresholds were lowest in the 1.2 to 1.4 range.  In addition, we conducted a local amplitude distribution analysis (i.e., assessed local a) with a large set of high resolution natural scene imagery and found the results of that analysis provided a better account of the a discrimination thresholds for stimuli presented to the fovea as opposed to the parafovea.

Influence of the Surround on Natural Scene Perception

Recent studies explored the sensitivity of human perception to natural images, in particular the sensitivity of the visual system to discriminate changes in the amplitude spectrum slope. Previous slope discrimination experiments were carried out with stimuli presented either in the fovea or the parafovea / periphery, and show that both yield poor discrimination at very steep or relatively shallow slopes. We verified if the well-known center-surround spatial interactions that operate early on in the visual processing stream influence the perception of real world images.  The results show that amplitude slope discrimination is greatly reduced (i.e., flat) when the stimulus is viewed in isolation. However, when a 2º target is placed within a surround containing an amplitude spectrum slope of 1 or 1.3, we see significant facilitation in detecting variations in the slope of the amplitude spectrum, particularly when the target contains an amplitude spectrum slope of 1 and 1.3. The results suggest that our visual system is sensitive to contextual interactions for stimuli that have the characteristics of natural images.

Biological Motion Perception in the Periphery

Biological motion perception, having both evolutionary and social importance, is performed by the human visual system with a high degree of sensitivity.  It is unclear whether peripheral vision has access to the specialized neural systems underlying biological motion perception, however given the motion component, one would expect peripheral vision to be, if not specialized, at least highly accurate in perceiving biological motion.  Here we show that the periphery can indeed perceive biological motion.  However, the periphery suffers from an inability to detect biological motion signals when they are embedded in dynamic visual noise.  We suggest that this peripheral deficit is not due to biological motion perception per se, but to signal/noise segregation. 

Horizontal Bias in Human Visual Processing Of Orientation in Natural Scenes

Many encoding mechanisms and processing strategies in the visual system appear to have evolved to better process the prevalent content in the visual world.  Here we examine the relationship between the prevalence of natural scene content at different orientations and visual ability for detecting oriented natural scene content.  Whereas testing with isolated gratings shows best performance at horizontal and vertical (the “oblique effect”), we report that when tested with natural scene content, performance is best at obliques and worst at horizontal (the “horizontal effect”).  The present analysis of typical natural scenes shows that the prevalence of natural scene content matches the inverse of this horizontal effect pattern with most scene content at horizontal, next most at vertical, and least at obliques.  We suggest that encoding of orientation may have evolved to accommodate the anisotropy in natural scene content by perceptually discounting the most prevalent oriented content in a scene, thereby increasing the relative salience of objects and other content in a scene when viewed against a typical natural background.